The receivers for voice-band modems employing Trellis-coded modulation (TCM), as described in the Article "channel coding with multi-level/phase signals" by G. Ungerboeck, published in IEEE Trans. Inform. Theory, vol. IT-28, January 1982, typically use a linear equalizer followed by a separate soft-decision Viterbi decoder which determines the coded sequence closest in Euclidean distance to the equalizer output samples. For channels with severe amplitude distortion, linear equalization leads to a substantial noise enhancement. In principle, this noise enhancement can be reduced by using a decision-feedback equalizer (DFE) in front of the TCM decoder. The feedback section of the equalizer subtracts the intersymbol interference (ISI) caused by symbols detected previously as described in the Article "Adaptive equalization" by S. Qureshi, published in Proc. IEEE, vol. 73, September 1985, provided correct decisions are available with delay zero. However, since zero-delay tentative decisions obtained from the TCM decoder are not reliable, the performance improvement is marginal.
The optimum receiver for estimating an uncoded data sequence in the presence of ISI and additive white-Gaussian noise consists of a whitened-matched filter followed by a Viterbi decoder which performs maximum-likelihood sequence decoding on the ISI trellis, as described in the Article "Maximum-likelihood sequence estimation of digital sequences in the presence of intersymbol interference" by G. Forney, published in IEEE Trans. Inform. Theory, vol. IT-18, May 1972. The state complexity of this trellis is given by 2.sup.mL, where L represents the number of relevant ISI terms in the output signal of the WMF, and m denotes the number of transmitted bits per data symbol. The sizable complexity of such a receiver for channels where L is large led to the development of prefiltering and channel truncation techniques which shape the channel response to a desired one of acceptably short duration.
In the Article "Adaptive channel memory truncation for maximum likelihood sequence estimation" by D. Falconer and F. Magee, published in Bell System Technical Journal, vol. 52, November 1973, a joint optimization of the desired impulse response and a linear prefilter have been investigated. In the Article "A maximum-likelihood sequence estimator with decision-feedback equalization" by W. Lee and F. Hill, published in IEEE Trans. Commun, vol. COM-25, September 1977, a decision-feedback equalizer (DFE) has been proposed instead of a linear prefilter. The error propagation effect of the DFE prefilter can then be mitigated by incorporating a decision-feedback mechanism into the sequence estimator.
Other suboptimum approaches have been studied which reduce complexity by simplifying the Viterbi decoder itself. For example, in the Article "Reduced-state Viterbi decoders for channels with intersymbol interference" by F. Vermeulen and M. Hellman, published in Conf. Rec. ICC'74, Mineapolis, June 1974, a reduced-state decoder is presented where only a certain number of survivor paths are retained. Also, it has recently been shown in the Article "Reduced-state sequence estimation with set-partitioning and decision-feedback" by V. Eyuboglu and S. Qureshi, published in IEEE Trans. Commun. vol COM-36, January 1988, that partial representation of ISI terms using the set-partitioning allows a further reduction of the state complexity.